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The purpose of this meta-analysis was to determine the pre-requisite for and the efficacy of inquiry-based 
learning to improve academic performance in contrast to traditional lessons. Randomised studies with 
pre- and post-test design with control and treatment groups were filtered with a priori inclusion criteria 
and analysed in Review Manager (2014). Out of the results in ERIC 13 studies were found, which show 
that inquiry-based learning can be more effective when the students and the teachers are well-prepared 
(Messner, 2009). The results of this meta-analysis of the studies between 2011 and 2017, as well as the 
theory in the literature, show that there is a positive effect size towards its regular implementation in 
primary (d= 0.67) and secondary schools (d = 0.81). Therefore, there should be more inquiry-based 
learning lessons taught in science and in other school subjects and the pre-requisite according to Messner 
(2009) have an effect of d = 0.88. 
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1. Introduction 


Inquiry-based learning has been officially introduced in German-speaking countries since Meyer's 
(2003) definition of it. The sequence of teaching in inquiry-based learning differs from traditional 
lessons (Sembill, 1996). According to Huber (2003) this teaching style includes: developing the 
questions and hypotheses, choosing and executing methods, examining and presenting the results. 
The learners autonomously work in an interdisciplinary project because they have to combine 
different subjects. This can be a problem, especially for younger learners. According to Kuhn and 
Pease (2008), as there are differences in acquiring skills, it is advisable to analyse the outcomes for 
primary and secondary students separately. In scientific disciplines, the method of inquiry-based 
learning has several links to the general scientific approach and, therefore, can also be used in 
numerous publications (ERIC, 2017). However, Kirschner, Sweller and Clark (2006) also point out 
that this way of learning can be too demanding from a cognitive perspective and can thus lead to 
frustration and possibly to lower academic achievement. The extent to which this learning style 
affects the performance of primary and secondary school students (Gagné & Driscoll, 1988) will be 
examined in this meta-analysis. Messner (2009), who defined the term “inquiry-based learning”, 
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also considers a preparatory unit for teachers and/or students as the pre-requisite of this learning 
style. Therefore, the following research questions are at the centre of this meta-analysis: 

1. What is the level of effect on academic performance in primary or secondary school classes of 
inquiry-based learning in science compared to traditional lessons? 

2. What is the effect size of inquiry-based learning on academic performance in existing studies, 
if the pre-requisite recommended by Messner (2009), a preparatory unit for inquiry-based learning 
for teachers and/or students, is satisfied? 


1.1. Inquiry-based Learning versus Traditional Lessons 


In traditional lessons, unlike in inquiry-based learning, the teacher is at the centre of the process 
(Gasser, 2004). The National Research Council (1996) in America publishes studies on how 
teachers should carry out these lessons. The current term “inquiry-based learning” is more precise 
than the original term “inquiry”. It has been enhanced with different learning styles and best 
practice examples. In German-speaking countries, the implementation took place later. Messner 
(2009) described this form of learning as a preliminary stage for scientific work which is project- 
oriented, research/ discovery-based and problem-oriented. Essential components of Reitinger's 
(2013) definition are cognitive interest, exploration, hypothesising, choice of method, discourse 
and publication of results. Such pre-scientific activity will be referred to as “inquiry-based 
learning” in this paper. Since experimentation is understood as a way of solving problems, 
inquiry-based learning is seen as essential for gaining scientific knowledge (Hammann, Phan, & 
Bayrhuber, 2008). Therefore, it is mostly represented in biology, physics and chemistry (ERIC, 
2017). 


1.2. Preparation Unit and Age 


According to Messner (2009), preparation for inquiry-based learning should be a pre-requisite for a 
successful project implementation. He recommends a practice-oriented coaching or training unit, 
where the learners and/or the teachers are prepared for this teaching concept. In their longitudinal 
study, Kuhn and Pease (2008) identified a preparatory unit of three years as beneficial in their 
study for primary school pupils. Although secondary school pupils were not specifically prepared 
for inquiry-based learning through the use of a preparatory unit, their approach outperformed 
primary school learners in the following areas: structured, logical and thoughtful approach, 
quantitative analysis and meaningful understanding of the data. However, they struggled to 
achieve a successful project outcome. Thus, a preparation unit for younger learners was useful 
because their scientific skills were not properly developed and were being improved in order to be 
well-prepared for their future lessons. To what extent age plays a role should, therefore, not be 
neglected in this work. As the older learners benefited from a preparation unit in relation to the 
outcome of the project, it is necessary to examine the age groups separately. 


1.3. Academic performance 


Gagné and Driscoll (1988) understand the performance that can be observed as the result of 
learning. In order to quantify the learning success of inquiry-based learning sessions, pupils are 
usually tested on what they have learned in the lesson. Although Erpenbeck and Rosenstiel (2003) 
presented more than 40 competence measurement procedures in their manual, this work focusses 
on studies that use tests at the end of the inquiry-based lesson because this way of testing is 
predominantly used in publications (Abd-El-Khalick et al., 2004). The positive effect of inquiry- 
based learning in general (Hetmanek, Knogler, & Chu, 2017; Reitinger, 2013) and on the academic 
performance in school has been empirically proven in several studies (Chang, Sung, & Lee, 2003, 
Fischer, Mitchell, & Alamo, 2007, Friedman & Heafner, 2007; Guisti, 2008; Ross, Morrisson, & 
Lowther, 2010). 
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2. Method 
2.1. Review of the Literature 


The effect size was calculated from studies in which pupils in the control group were taught in a 
traditional lesson and in which pupils in the treatment group were engaged in an inquiry-based 
learning lesson. The search was conducted using ERIC (2017) with the following keywords: 
“inquiry learning, pretest, posttest, effectiveness, outcome” and using Google Scholar (2017) 
adding the words "inquiry-based learning, -flipped -game -maps -deaf -blind”. The time limit was 
drawn after the international project PRIMAS (2011) in order to present a current effect size. 
Additionally, it was possible to include European studies since a common definition of inquiry- 
based learning emerged after 2011 (Reitinger, 2013). All previous publications, as well as all 
studies that did not fit into the topic, that did not test pupils’ academic performance or that did not 
have a control group or were published without a peer review, were excluded. 


2.2. Selection Criteria 


After reviewing the studies, they were analysed to determine if they fitted the selection criteria 
such as quantitative studies, articles from journals and dissertations since 2011, research results 
from primary and secondary schools in science, studies with an internal consistency of Cronbach’s 
a> 0.6 (Hinton, Brownlow, McMurray, & Cozens, 2004, p. 365), the treatment group was taught 
according to the definition by Reitinger (2013) and the control group to the definition of Gasser 
(2004, p. 141). 


2.3. Included Studies 


A total of 338 studies were found (319 studies from Google Scholar, 18 articles from ERIC and one 
study from other sources, a book by Reitinger (2013)) of which 261 were excluded after review of 
the abstract and a further 65 after reviewing the full text. Reasons for this decision were: they 
didn’t meet the inclusion criteria, the learners were older than 18 years or the data was 
inappropriate for a meta-analysis. No duplicate publications were found. 6 Studies from ERIC and 
7 from Google Scholar provided a complete data set. 

Table 1 shows the studies with the year of publication and the country of investigation and the 
primary or secondary level. Furthermore, the maximum number of 13 points according to 
Luborsky, Singer and Luborsky (1975) is given in brackets for the quality of how the study was 
carried out. These are the examined studies for the meta-analysis: Abdi (2014), Barak, Ashkar and 
Dori (2011), Barthlow (2011), Chiang, Yang and Hwang (2014), Dijk and Lazonder (2013), Ergiil, 
Simsekli, Calis, Ozdilek, Gocmencelebi and Sanli (2011), Hasan (2012), Hashim, Ababkr and Eljack 
(2015), Maxwell, Lambeth and Cox (2015) , Meij, Meij, & Harmsen (2015), Njoroge, Changeiywo 
and Ndirangu (2014), as well as Omokaadejo (2015). 


2.4. Coding 


The coding was carried out according to: 

* the names of the authors with publication year and country 

* the age of the learners in secondary (S) or primary/elementary (E) school 

* the quality characteristics of the study (Luborsky, Singer, & Luborsky, 1975), and 


* a preparatory unit or a practical training session for inquiry-based learning session according to 
the definition of Messner (2009), in which teachers and/or students are prepared to carry out or 
participate successfully in an inquiry-based learning lesson. 
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Table 1. 
Characteristics of included studies 


Elementary (E)/ Preparation to 


Author Year Country Secondary (5) Messner (2009) (quality) 
Abdi 2014 Iran E - 12 
Barak 2011 Israel E - 13 
Barthlow 2011 USA oS Yes 13 
Chiang 2014 Taiwan E - 12 
Dijk 2013 Netherlands S - 12 
Ergiil 2011 Turkey E Yes Aa 
Ergiil 2011 Turkey 5 Yes 13 
Hasan 2012 Dubai S - 12 
Hashim 2015 Nigria S - 13 
Maxwell 2015 USA E - 12 
Meij 2014 Netherlands E - IZ 
Njoroge 2015 Kenia S, Yes 13 
Omokaadejo 2015 Nigeria 5 - 13 


2.5. Calculation of the results 


A random-effect meta-regression was calculated using the statistical programme Review Manager 
(2014). Entering the mean and the standard deviation for the treatment and control groups 
provided a weighted effect size and a Cl of 95% for each study and subgroup. There were no 
missing data in the calculation. The Q-test shows the heterogeneity of the studies. The mean 
difference and IV for the inverse variance were chosen as an output mode because academic 
performance is a continuous dependent variable (Higgins & Green, 2011). Cohen (1988) interprets 
a value of <0 as negative, a value of 0.0 - 0.2 as none, from 0.21 - 0.5 as a small, from 0.51 - 0.8 as an 
average and from 0.8 as a large effect. 


2.6. Studies 


Eight studies were conducted in secondary and five in primary schools. Inquiry-based learning is 
carried out according to the definition of Messner (2009) and Reitinger (2013) and described as 
follows in the studies: learners should engage in self-directed activities (Abdi, 2014), gather and 
discuss results from the following (virtual) experiments on: electrical circuit (Njoroge, 
Changeiywo, & Ndirangu, 2014), physical constructs (Meij, Meij, & Harmsen, 2015), atomic 
constellations (Barthlow, 2011) , underwater scenarios (Chiang, Yang, & Hwang, 2014), prey 
species (Hasan, 2012), cycle of nature (Barak, Ashkar, & Dori, 2011), greenhouse effect (Dijks & 
Lazonder, 2013) and in science in general (Ergiil, Simsekli, Calis, Ozdilek, Gocmencelebi, & Sanli, 
2011) A standardised test has been used in the studies by Abdi (2014), Hasen (2012), Njoroge, 
Changeiywo and Ndirangu (2014) and Ergiil, Simsekli, Calis, Ozdilek, Gocmencelebi and Sanli 
(2011). 


2.7. Quality of Results 


To ensure the quality of the results of the procedure, an 18-item checklist according to the 
QUOROM method by Moher et al. (1999) was added to Appendix 1. The studies of Abdi (2014), 
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Chiang, Yang and Hwang (2014), Dijk and Lazonder (2013), Ergiil, Simsekli, Calis, Ozdilek, 
Gocmencelebi and Sanli (2011), Hasan (2012), and Max and Meij, Meij and Harmsen (2015) 
achieved only 12 instead of 13 quality points for the quality of their performance according to 
Luborsky, Singer and Luborsky (1975) because the sample size was smaller than in the other 


studies. A funnel plot (as in Figure 1) can detect a possible publication bias that affects the quality 
of the research. 
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Figure 1. Funnel plot 


If the funnel plot is symmetrical and similar to a pyramid, the bias is lower (Suchmacher & 
Geller, 2012). According to Ressing, Blettner and Klug (2009), the appearance of studies in the 
lower left of the triangle, which are visible in this research funnel plot, is an indication of no bias. 


3. Results 


The following forest plots represent the results and show the first author, the year of publication, 
the country, primary/elementary (E) or secondary (S) school, the quality of the study in brackets, 
the mean values, the standard deviation for the treatment (experimental) and the control group, 
the weight of the study, the heterogeneity, the number of the sample size and the individual and 
the overall effect size below the studies with a confidence interval of 95%. A random-effect model 
was chosen because the studies suggest heterogeneity. 


Table 2 shows the mean differences in pupils’ academic performance in inquiry-based learning 
versus traditional lessons. 


Table 1. 
Forest plot for all studies 
Experimental Control Std. Mean Difference Std. Mean Difference 
Study or Subgroup Mean SD Total Mean SD Total Weight IV, Fixed, 95% Cl IV, Fixed, 95% Cl 
Abdi 2014 | Iran E |N |(12) 7.3 0.979 20 6.35 1.226 20 1.5% 0.84 [0.19, 1.49] 
Barak 2011 | Israel E |r |(13) 70.34 21.78 926 55.17 19.54 409 44.4% 0.72 [0.60, 0.84] 
Bartholow 2011 |USA S jr |(13) 14.6 3.573 150 11.64 3.798 171 12.2% 0.80 [0.57, 1.03] 
Chiang 2014 | Taiwan E |r |(12) 80.14 8.763 28 73.93 13.703 29 2.3% 0.53 [0.00, 1.06] 
Dik 2013 [Netherlands S (I |(12) 41.44 20.31 19 33.54 20.28 23 1.7% 0.38 [-0.23, 1.00] 
Ergill 2011 | Turkey E |N |(12) 14.042 2.582 10 12.22 3.352 12 0.9% 0.58 [-0.28, 1.44] 
Ergiil 2011 | Turkey SN |(13) 14.6 3.573 150 11.64 3.798 171 12.2% 0.80 [0.57, 1.03] 
Hasan 2012 | Dubai S |W (12) 14.84 5.26 36 «12.1 5.76 16 1.8% 0.50 [-0.10, 1.10] 
Hashim 2015 | Nigria S |N |(13) 18.776 5.265 101 15.13 2.16 101 7.6% 0.90 [0.61, 1.19] 
Maxwell (2015 |USA E |N |(12) 78.82 15.73 42 74.95 17.45 42 3.4% 0.23 [-0.20, 0.66] 
Meij 2014 Netherlands E I |(12) 12.666 4.8749 10 12.65 4.2951 11 0.9% 0.00 [-0.85, 0.86] 
Norjorge (2015 | Kenia S |W (13) 22.47 6.46 96 15.51 4.74 87 6.3% 1.21 [0.90, 1.53] 
Omocadeco 2008 Nigeria S |N |{13) 52.74 8.5 60 48.51 8.47 60 4.8% 0.50 [0.13, 0.86] 
Total (95% Cl) 1648 1152 100.0% 0.74 [0.66, 0.82] 


Heterogeneity: Chi? = 23.25, df = 12 (P = 0.03); I? = 48% 


-1 
Test for overall effect: Z = 18.12 (P < 0.00001) { eperimental] Favours [control] 
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The positive values of the mean differences indicate a higher level of academic performance in 
inquiry-based learning lessons as opposed to traditional lessons. I? is <50% in this study. Higgins, 
Thompson, Deeks and Altmann (2003, p. 559) consider this value of heterogeneity to be low and 
thus the weighted effect size can be interpreted. 

The following results in Table 3 were calculated to answer the research question regarding the 
learners’ age. 


Table 3. 
Forest plot for studies in secondary schools 

Experimental Control Std. Mean Difference Std. Mean Difference 
Study or Subgroup Mean SD Total Mean SD Total Weight IV, Fixed, 95% Cl IV, Fixed, 95% Cl 
Bartholow |2011 (USA S jf |(13) 14.6 3.573 150 11.64 3.798 171 26.2% 0.80 [0.57, 1.03] “= 
Dijk 2013 [Netherlands S [I (12) 41.44 20.31 19 33.54 20.28 23 3.6% 0.38 [-0.23, 1.00] 
Ergiil 2011 | Turkey S _|N \(13) 14.6 3.573 150 11.64 3.798 171 26.2% 0.80 [0.57, 1.03] = 
Hasan 2012 | Dubai SW (12) 14.84 5.26 36 0=«12.1 5.76 16 3.8% 0.50 [-0.10, 1.10] =. 
Hashim 2015 _| Nigria S\N \(13) 18.776 5.265 101 15.13 2.16 101 16.2% 0.90 [0.61, 1.19] - -% 
Norjorge 2015 | Kenia S |W (13) 22.47 6.46 96 15.51 4.74 87 13.6% 1.21 (0.90, 1.53] = 
Omocadco (2008 | Nigeria S |N \(13) 52.74 8.5 60 48.51 8.47 60 10.3% 0.50 [0.13, 0.86] ———— 
Total (95% Cl) 612 629 100.0% 0.81 [0.70, 0.93] ed 
Heterogeneity: Chi? = 12.48, df = 6 (P = 0.05); I? = 52% + + ry 7 $ 


Test for overall effect: Z = 13.68 (P < 0.00001) [experimental] Favours [control] 


An effect size of 0.81 is calculated for the studies carried out in secondary schools 
(TG n = 612, KG n = 629, 0.81 with 95% CI of [0.70, 0.93], p = 0.001). These seven studies are slightly 
above the 50% threshold of homogeneity (Chi? = 12.48, df = 6 (p = 0.05), ? = 52%). 

In contrast, the effect size of 0.67 is lower in the studies in primary schools (TG n = 1036, 
KG n = 523, 0.67 with 95% CI of [0.56, 0.78], p = 0.001) than in secondary schools. These six studies 
are homogeneous (Chi 2 = 7.54, df = 5 (p = 0.18), 1 2 = 34%). The effect sizes are 0.81 for secondary 
schools and 0.67 for primary schools. This outcome is gathered from studies found between 2011 
and 2017 which compare the learners' academic performance between inquiry-based learning and 
traditional lessons. The outcome of these studies suggest that inquiry-based learning is a suitable 
teaching method in primary and secondary schools in order to increase pupils’ academic 
performance in science. 


Table 4. 
Forest plot for studies in primary schools 
Experimental Control Std. Mean Difference Std. Mean Difference 
Study or Subgrou Mean SD Total Mean SD Total Weight IV, Fixed, 95% Cl IV, Fixed, 95% Cl 
Abdi 2014 Iran E |N |(2) 7.3 0.979 20 6.35 1.226 20 2.8% 0.84 (0.19, 1.49] 
Barak 2011 Israel E |E (13) 70.34 21.78 926 55.17 19.54 409 83.2% 0.72 (0.60, 0.84] 
Chiang 2014 Taiwan E |f |(2) 80.14 8.763 28 73.93 13.703 29 4.2% 0.53 (0.00, 1.06] 
Ergiil 2011 | Turkey EN (12) 14.042 2.582 10 12.22 3.352 12 1.6% 0.58 [-0.28, 1.44] 
Maxwell [2015 {USA E |N |(12) 78.82 15.73 42 74.95 17.45 42 6.5% 0.23 [-0.20, 0.66] 
Meij 2014 |Netherlands E I (12) 12.666 4.8749 10 12.65 4.2951 11 1.6% 0.00 [-0.85, 0.86] 
Total (95% Cl) 1036 523 100.0% 0.67 [0.56, 0.78] 


Heterogeneity: Chi? = 7.54, df = 5 (P = 0.18); I? = 34% 


Test for overall effect: Z = 12.01 (P < 0.00001) - 


[experimental] Favours [control] 


According to Messner (2009, p. 25ff), a preparatory unit for teachers/and for students in 
inquiry-based learning settings is required such as that used in the study by Njoroge, Changeiywo 
and Ndirangu (2014). A preparatory unit for teachers was carried out in a study by Barthlow 
(2011). There was also an introduction for teachers and learners in the study by Ergiil, Simsekli, 
Calis, Ozdilek, Gocmencelebi and Sanli (2011). The effect size of 0.88 (TG n = 406, KG n = 441, 0.88 
with 95% CI of [0.73, 1.02], p = 0.001) of these studies is very high and they are homogeneous 
(Chi? = 5.73, df =3 (p = 0.13), I? = 48%). 
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Table 5. 
Forest plot for studies with a preparation unit 
Experimental Control Std. Mean Difference Std. Mean Ditterence 
Study orSubgroup  ————s Mean SD Total Mean SD Total Weight IV, Fixed, 95% Cl IV, Fixed, 95% Cl 
Bartholow 2011 USA Sit (13) 14.6 3.573 150 11.64 3.798 171 38.6% 0.80 0.57. 1.031 
Exgiil 2011 Turkey EN (2) 14.042 2.582 10 12.22 3.352 12 2.7% 0.58 [-0.28, 1.44] 
Exgiil 2011 Turkey SN (3) 14.6 3.573 150 11.64 3.798 171 38.6% 0.80 [0.57, 1.03] 
Nonorge 2015 Kenia S |W (3) 22.47 6.46 96.15.51 4.74 87 20.1% 1.21 [0.90, 1.53] 
Total (95% Cl) 406 441 100.0% 0.88 [0.73, 1.02] 


Heterogeneity: Chi? = 5.73, df = 3 (P = 0.13); ? = 48% 
Test for overall effect: Z = 12.13 (P < 0.00001) 


-2 - 1 
s [experimental] Favours [control] 


The effect size of the studies found between 2011 and 2017, which compared pupils’ academic 
performance in inquiry-based learning with traditional lessons was 0.67 in primary schools and 
0.81 in secondary schools. If a preparation unit was carried out, a pre-requisite recommended by 
Messner (2009), the effect size of these studies was 0.88. One reason for this high value could be 
that inquiry-based learning is a common method in the scientific disciplines (ERIC, 2017) and it is 
more established than in other school subjects. 


4. Tendency, Comment and Conclusion 


The examined studies show a favorable tendency towards improved learning as a result of 
inquiry-based learning over traditional lessons. The effect size of inquiry-based learning is higher 
in secondary than in primary education. The highest effect size was seen when a preparation unit 
for the inquiry-based learning lesson is present. An area of future investigation in this field would 
be to examine the effect sizes of inquiry-based learning lessons with digital tools and how they can 
be successfully used in this process. This meta-analysis encourages teachers to offer inquiry-based 
learning sessions at school and in school subjects other than science. This is how researchers 
should be able to find sufficient opportunities in the future to explore all areas, especially in artistic 
and creative school subjects. This meta-analysis shows that there are insufficient studies carried 
out with a pre- and post-test design and with control and experimental groups. Studies are rare in 
primary schools and therefore more research should be carried out in this field. The limitation of 
this study was the small number of examined studies and of studies which showed negative 
results. For that reason, dissertations were also included and they showed no or little effect sizes, 
which provides more balance to this research. Consequently, a random-effect model was used 
because it also takes unpublished outcomes into consideration. This work corresponds to other 
small-scale (Furtak, Seidel, Iverson, & Briggs, 2012) meta-studies that are valuable in gaining 
insight in this research field overall. An implementation of inquiry-based learning at a young age 
is therefore justified. Since learners should take on more and more responsibility for their learning 
success and process (Bastiaens, Schrader, & Deimann, 2016), educators are required to try out new 
forms of learning in order to meet the demands of society. Inquiry-based learning supports science 
education. If learners and teachers have an opportunity to receive a preparatory introduction in 
accordance with the recommendations of Messner (2009) to this pre-scientific teaching method, it 
is even more successful, especially in primary schools. 
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